Improving Interpretability and Regularization in Deep Learning

Improved Interpretability and Generalisation for Deep Learning

Beyond Sparsity: Tree Regularization of Deep Models for Interpretability

The lack of interpretability remains a key barrier to the adoption of deep models in many applications. In this work, we explicitly regularize deep models so human users might step through the process behind their predictions in little time. Specifically, we train deep time-series models so their classprobability predictions have high accuracy while being closely modeled by decision trees with ...

Spectral Norm Regularization for Improving the Generalizability of Deep Learning

We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades the performance on it. To reduce the sensitivity to perturbation, we propose a simple and effective regularization method, referred to as spectral norm regularization, which penalizes the high spectral norm of weight m...

Deep learning - Regularization

where, θ̃ is an estimator of θ coming from update equations or solution of optimization procedure. Variability in θ̃ is because of randomness in data and bias is due to model mismatch. Well known bias-variance trade offAs complexity of the model is increased model mismatch(bias) is decreased while variance in the prediction is increased because of randomness in training inputs. 4. Deep Learning s...

Implicit Regularization in Deep Learning

In an attempt to better understand generalization in deep learning, we study several possible explanations. We show that implicit regularization induced by the optimization method is playing a key role in generalization and success of deep learning models. Motivated by this view, we study how different complexity measures can ensure generalization and explain how optimization algorithms can imp...